Mutation testing from probabilistic finite state machines

Mutation testing traditionally involves mutating a program in order to produce a set of mutants and using these mutants in order to either estimate the effectiveness of a test suite or to drive test generation. Recently, however, this approach has been applied to specifications such as those written as finite state machines. This paper extends mutation testing to finite state machine models in which transitions have associated probabilities. The paper describes several ways of mutating a probabilistic finite state machine (PFSM) and shows how test sequences that distinguish between a PFSM and its mutants can be generated. Testing then involves applying each test sequence multiple times, observing the resultant output sequences and using results from statistical sampling theory in order to compare the observed frequency of each output sequence with that expected

Testing from a stochastic timed system with a fault model

In this paper we present a method for testing a system against a non-deterministic stochastic finite state machine. As usual, we assume that the functional behaviour of the system under test (SUT) is deterministic but we allow the timing to be non-deterministic. We extend the state counting method of deriving tests, adapting it to the presence of temporal requirements represented by means of random variables. The notion of conformance is introduced using an implementation relation considering temporal aspects and the limitations imposed by a black-box framework. We propose an algorithm for generating a test suite that determines the conformance of a deterministic SUT with respect to a non-deterministic specification. We show how previous work on testing from stochastic systems can be encoded into the framework presented in this paper as an instantiation of our parameterized implementation relation. In this setting, we use a notion of conformance up to a given confidence level

Extending stream X-machines to specify and test systems with timeouts

Stream X-machines are a kind of extended finite state machine used to specify real systems where communication between the components is modeled by using a shared memory.In this paper we introduce an extension of the Stream X-machines formalism in order to specify delays/timeouts.The time spent by a system waiting for the environment to react has the capability of affecting the set of available outputs of the system. So, a relation focusing on functional aspects must explicitly take into account the possible timeouts.We also propose a formal testing methodology allowing to systematically test a system with respect to a specification. Finally, we introduce a test derivation algorithm. Given a specification, the derived test suite is sound and complete, that is, a system under test successfully passes the test suite if and only if this system conforms to the specification

Using genetic algorithms to generate test sequences for complex timed systems

The generation of test data for state based specifications is a computationally expensive process. This problem is magnified if we consider that time con- straints have to be taken into account to govern the transitions of the studied system. The main goal of this paper is to introduce a complete methodology, sup- ported by tools, that addresses this issue by represent- ing the test data generation problem as an optimisa- tion problem. We use heuristics to generate test cases. In order to assess the suitability of our approach we consider two different case studies: a communication protocol and the scientific application BIPS3D. We give details concerning how the test case generation problem can be presented as a search problem and automated. Genetic algorithms (GAs) and random search are used to generate test data and evaluate the approach. GAs outperform random search and seem to scale well as the problem size increases. It is worth to mention that we use a very simple fitness function that can be eas- ily adapted to be used with other evolutionary search techniques

Passive Testing with Asynchronous Communications and Timestamps

We develop a formal passive testing framework for software systems where parties communicate asynchronously. Monitors, placed in between the en- tities, check that a certain property holds over the observations of the interaction between users and the System Under Test (SUT). Due to the asynchronous nature of communications, the trace observed by the monitor might differ from the one produced by the SUT: the monitor observes inputs before they are received by the SUT and outputs are observed after they are sent by the SUT. It is neces- sary to take this into account in passive testing; otherwise we might obtain false positives or false negatives. In order to better assess the real causality between actions, we consider the case where each action is labelled with a timestamp giv- ing the time when it was observed at the monitor. We also assume that we know bounds on network latency and so the timestamps allow us to determine additional causalities between actions. Our monitors are implemented as automata that take into account communications being asynchronous. Our solution checks properties against traces in polynomial time and has low storage requirements. Therefore, our proposal is suitable for real-time passive testing.Research partially supported by the projects DArDOS (TIN2015-65845-C3-1-R (MINECO/FEDER)) and SICOMORo-CM (S2013/ICE-3006)

An extended framework for passive asynchronous testing

In passive testing a monitor observes the trace (sequence of inputs and outputs) of the system under test (SUT) and checks that this trace satisfies a given property $P$, potentially triggering a response if an incorrect behaviour is observed. Recent work has explored a variant of passive testing, in which we have a required property $P$ of the traces of the SUT and there is a first-in-first-out (FIFO) network between the SUT and the monitor. The problem here is that the trace observed by the monitor need not be that produced by the SUT. Previous work has shown how such asynchronous passive testing can be performed if the property $P$ is defined by a pair $(\rho,O_{\rho})$ that represents the requirement that if trace $\rho$ is produced by the SUT then the next output must be from the set $O_{\rho}$. This paper generalises the previous work to the case where the property $P$ is defined by a finite automaton

A tool supported methodology to passively test asynchronous systems with multiple users

Spanish MINECO/FEDER (grant number TIN2015-65845-C3-1-R); Region of Madrid (grant number S2013/ICE-3006)

Integration testing of heterotic systems

Computational theory and practice generally focus on single-paradigm systems, but relatively little is known about how best to combine components based on radically different approaches (e.g. silicon chips and wetware) into a single coherent system. In particular, while testing strategies for single-technology artefacts are generally well developed, it is unclear at present how to perform integration testing on heterotic systems: can we develop a test-set generation strategy for checking whether specified behaviours emerge (and unwanted behaviours do not) when components based on radically different technologies are combined within a single system? In this paper, we describe an approach to modelling multi-technology heterotic systems using a general-purpose formal specification strategy based on Eilenberg's X-machine model of computation. We show how this approach can be used to represent disparate technologies within a single framework, and propose a strategy for using these formal models for automatic heterotic test-set generation. We illustrate our approach by showing how to derive a test set for a heterotic system combining an X-machine-based device with a cell-based P system (membrane system)

Intraperitoneal drain placement and outcomes after elective colorectal surgery: international matched, prospective, cohort study

Despite current guidelines, intraperitoneal drain placement after elective colorectal surgery remains widespread. Drains were not associated with earlier detection of intraperitoneal collections, but were associated with prolonged hospital stay and increased risk of surgical-site infections.Background Many surgeons routinely place intraperitoneal drains after elective colorectal surgery. However, enhanced recovery after surgery guidelines recommend against their routine use owing to a lack of clear clinical benefit. This study aimed to describe international variation in intraperitoneal drain placement and the safety of this practice. Methods COMPASS (COMPlicAted intra-abdominal collectionS after colorectal Surgery) was a prospective, international, cohort study which enrolled consecutive adults undergoing elective colorectal surgery (February to March 2020). The primary outcome was the rate of intraperitoneal drain placement. Secondary outcomes included: rate and time to diagnosis of postoperative intraperitoneal collections; rate of surgical site infections (SSIs); time to discharge; and 30-day major postoperative complications (Clavien-Dindo grade at least III). After propensity score matching, multivariable logistic regression and Cox proportional hazards regression were used to estimate the independent association of the secondary outcomes with drain placement. Results Overall, 1805 patients from 22 countries were included (798 women, 44.2 per cent; median age 67.0 years). The drain insertion rate was 51.9 per cent (937 patients). After matching, drains were not associated with reduced rates (odds ratio (OR) 1.33, 95 per cent c.i. 0.79 to 2.23; P = 0.287) or earlier detection (hazard ratio (HR) 0.87, 0.33 to 2.31; P = 0.780) of collections. Although not associated with worse major postoperative complications (OR 1.09, 0.68 to 1.75; P = 0.709), drains were associated with delayed hospital discharge (HR 0.58, 0.52 to 0.66; P < 0.001) and an increased risk of SSIs (OR 2.47, 1.50 to 4.05; P < 0.001). Conclusion Intraperitoneal drain placement after elective colorectal surgery is not associated with earlier detection of postoperative collections, but prolongs hospital stay and increases SSI risk

Bounded reordering in the distributed test architecture

In the distributed test architecture, the system under test interacts with its environment at multiple physically distributed ports and the local testers at these ports do not synchronise their actions. This presents many challenges and, in particular, apparently incorrect behaviours can be the consequence of an erroneous assumption about the exact order in which actions were performed at different ports. In previous work, we defined a conformance relation for the distributed test architecture. Essentially, the system under test is faulty if we observe a trace such that no admissible reordering of the actions in could have been produced by the specification. However, this notion can be weak if the compared traces might be too different. This paper introduces conformance relations where, for a given metric, a reordering is only considered if the distance between the two traces is at most a certain bound k. We introduce two different metrics and provide algorithms to construct finite automata accepting these close, with respect to each metric, sequences. We also study the computational complexity of the two main problems associated with the new framework: deciding whether a trace is accepted by the new automaton and deciding whether one system conforms to a specification with respect to the new conformance relation